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INSPECTOR GENERAL 

DEPARTMENT OF DEFENSE 
4800 MARK CENTER DRIVE 
ALEXANDRIA, VIRGINIA 22350-1500 


Ref: FOIA-2013-00686 


Dear Mr. Greenewald: 

This is in response to your September 9, 2013, Freedom of Information Act (FOIA) 
request for a copy of all documents pertaining to the DoD IG’s investigation of the NSA 
TR ATT, BLAZER Program. We received your request on September 11, 2013, and assigned it 
case number FOIA-2013-00686. 

The Office of the Deputy Inspector General for Intelligence and Special Program 
Assessments conducted a search and located the enclosed report. I determined that the redacted 
portions are exempt from release pursuant to 5 -U.S.C. § 552(b)(1), which protects from 
disclosure of national security information concerning the national defense or foreign policy; 5 
U.S.C. § 552(b)(3), which covers information specifically exempted from disclosure by statute; 
and 5 U.S.C. § 552(b)(5), which protects inter-agency or intra-agency memorandums or letters 
which would not be available by law to a party other than an agency in litigation with the 
agency. 

If you are not satisfied with this action, you may submit an administrative appeal to the 
Department of Defense, Office of Inspector General, ATTN: FOIA Appellate Authority, Suite 
10B24, 4800 Mark Center Drive, Alexandria, VA 22350-1500. Your appeal should be 
postmarked within 30 days of the date of this letter, should cite to case number FOIA-20 13- 
00686, and should be clearly marked “Freedom of Information Act Appeal.” 

Sincerely, 

Catherine M. DelPrete 
Supervisory Attorney 
Freedom of Information, Privacy and 
Civil Liberties Office 

Enclosure(s): 

As stated 
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Additional Copies (U) 

(V To F. Gimble^^ 

Suggestions for Audits (U) 

(U) To suggest ideas for or to request future audits or evaluations of Defense 
intelligence issues, contact the Office of the Deputy Inspector General for 
Intelligence at (703) 604-8800 (DSN 664-8800) or fax (703) 604-0045. Ideas and 
requests can also be mailed to: 

(U) 

Office of the Deputy Inspector General for Intelligence 
Attn: Executive - Officer 
Inspector General of the Department of Defense 
400 Army Navy Drive (Room 703) 

Arlington, VA 22202-4704 


(U) 


Defense Hotline (U) 

(U) To report fraud, waste, or abuse, contact the Defense Hotline by calling 
(800) 424-909.8; by sending an electronic message to Hotline@dodig.osd.mil : or 
writing the Defense Hotline, The Pentagon, Washington, DC 20301-1 900. The 
identity of each writer and caller is fully protected. 


Acronym (U) 

(U) NSA/CSS National Security Agency/Central Security Sendee 
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December 15, 2004 

MEMORANDUM FOR DIRECTOR, NATIONAL SECURITY AGENCY / CHIEF, 
CENTRAL SECURITY SERVICE (U) 

SUBJECT: Audit of the Requirements for the TRAILBLAZER and THINTHREAD 
Systems (Report No. 05-1NTEL-03) (Project No. D2003AL-Q1G0) (U //FOUQ) 


(U) We are providing this report for review and comment. This audit was 
initiated from a Hotline complaint. We considered management comments on a draft of 
this report in preparing the final report. Management conducted a security classification 
review of the draft report. - 

(U) DoD Directive 7650.3 requires that all recommendations be resolved 
promptly. Management nonconcurred with Recommendations A.l. and A.2. 
Management partially concurred with Recommendations A.3., B.I., B.2., and B.3. As a 
result of management comments, we revised Recommendations A.2.and B.I . We request 
that the Director, National Security Agency/Central Security Service provide additional 
comments on all recommendations by January 1 5, 2005. 

(U ) We appreciate the courtesies e xtended t o the staff. Questions should be 
directed (703) 604-H|or 

(41 0) 854-HH See Appendix K for the report distribution. The team members are 
listed inside the back cover. 
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Office of the Inspector General of the Department of Defense / 

December 15, 2005 


Report No. 05-INTEL-O3 
(Project No. D2003AI/-0I0Q) 


Requirements for the TRAILBLAZER and THINTHREAD 
Systems (U) / 

■ , . / 

Executive Summary (U) 



/ (U//Et)tJ©).Backgrouad. We performed this audit in response to a Defense Hotline 
■ complaint, which alleged that NSA actions in the development pf THINTHREAD and 
TRAILBLAZER resulted in fraud, wa ste, and abuse. The spec ific allegations were: (1) 

Jdollars; (2) NSA 


TRAILBLAZERdevelppraent waste 




(P/FWQ) TRAILBLAZER. TRAILBLAZER is an Acquisition Cafegoty-lA progr 

to acquire, through a series of spiral developments, an integrate 4 

much-needed mission capability against the Global Network.- L 

|TRAILBLAZER is being / 

developed. See Appendix B tor turther -details. 

(TJ/lTOgoj __ I 


J— 


(b)(3)-P.L. 86-36 
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(uz/FmeiT 


\\ 


\ 


./ 


flgMResults. NSA enhanced existing digital network exploitation systems,] 


_} the National^ Security \ 


Agency is inefficiently using resources to develop a digital network exploitation system 
that is not capable of folly exploiting the digital net work intelligence available to r — 1 — *• 
from the Global Information Network. NSA should I 


As a result, theNSA transformation effort may be developing a Jess capable long-term 
digital network exploitation solution that will take longer and cost significantly more to 
^ develop. NSA management should designate a team.to a 

(bHaT^L^ss^s ~ exploitation requirements of the TRAILBLAZER Technology Demonstration Platform. 
in (5) ——Also, USA management should provide the results of this assessment to the 

^V\. TRAJLBLAZER~TechnoIogy Demonst ration Platform Milestone Decision Authority for 

\ N<; s n ^ use in n ‘ ~ r 

■V\% 



\\ ' 


X <TO Management Gojnments and Audit Response. The management comments form 
\'N, \the Dkector,'NSjVChief; Geptral Security Service contained significant disagreements 

hii<h r^rLArf ■firtitinoo TTl»»T\Jr/»/*tnr ctofp/1 ftiof fh« u/nrtrl’o rnnet arliinnrw? fnmfnlnfri 


with the report findings. The'Direptor stated that the world’s most advanced c. 

\\ experts have assessed '^situation arid are confi dent in the approach NSA has taken. The 

' . 4 director non concurred witfrthe re^mmeridattor)/ 


\\ \ \ concurred or 

\\ A “ 


-JThe Director 7 


•and u ^ 

\ Program Executive 0 


rent as part of the TRAILBLAZER Milestone B decision. The 

Leer and TRAILBLAZER management will estimate the 

. . impacts after they receive estimates from the review team and will decide 

\\ \ at that time whether to concur. Three of the four partial concurrences depend upon the 
Y\ \ Program Executive Officer and TRAILBLAZER management decisi&rf 

\\ . ‘I 

AY 


_V 


tsr \ \ "(TO'We revised thd report findings and reco mmendations based on the Director’s 

y\, comments. Howevefr we, con tinue toJbolicvtj_ 


1 We request that the NSA/Chief, Central 


t J We do n ot agree with management tnat it would be detrimental to tlie 

signals intelligence mission ] | 


TOP SECREl , v yee afl gW20291123 



3 25 - 3 - 3 - 4 - 6 - 


(U)j 1 V/e request that the Director, 

NSA/Chief, Central Security Service provide additional comments to this report by 
January 15, 2005. See the Findings section of the report for a discussion of management 
comments and audit responses, Appendix J for a detailed audit response to the 
management comments, and the management comments section of the report for the 
complete text of the comments 
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National Security Agency, Central Security Agency Comments 1 14 



(U//FOUQ-) 
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(U) Background 


^SyBatime Allegations. A letter to the Defense Hotline request that the 
Inspector General of the D epartment of Defense audit and investigat e the 
acquisition of the TRAIL-BLAZER and THINTHREAD prograrns.\The'complaint 
charged the NSA with fraud, waste, abuse, and a disr egard fur a solution tb u rgent 
security needs because the TRAILBLAZER ProgramT \ \ 


(U) National Security Agency Mission and Planning Guidance. The mission 
of the National Security Agency/Central Security Service (NSA/CSS) is to 
provide actionable Signals Intelligence to U.S. decisionmakers, from the national 
to the tactical level, while protecting the nation’s vital information from attacks. 
The urgency of the NSA/ CSS mission is greater than ever since the September 
1 1th attacks and the Global War on Terrorism. 

(U/7FpLf©Jin addition, defense planning guidance and the Director of Central 
Intelligence guidance stipulate the need for “a globally vigilant intelligence 
system that can provide early strategic warning of crisis and detect threats” that is 
supported by capabilities such as an “information superiority backbone.” 

_(gffSL)-TRAILBLAZER is ^ { major acquisition program, which 

started in FY 2000 and is scheduled tor initial operating capability in FY 2009, to 
acquire an integrated system that is composed largely of commercial technology 
to provide the much-needed mission capability against the global network through 
a series of spiral developments. The TRAILBLAZER system will deliver 
hardware and software, system integration, and program integration services to 
include: - 




TETT31~gA— 3. 6-36 


1B^{g^55TET~«G-3S— ... ___ — _ 



(ID Policy GoVerningf 


~}\ cqiii3itfoii. Guidance for 
Requisition is provided in DoD Directive 5000.1 , 


DoD instruction OUOO.Z, and the Clinger Cohen Act of 1 996. Specifically, these 
regulations provide the framework for the acquisition of major automated 


(U) DoD Directive 5000.1, “The Defense Acquisition System,” May 12, 2003. 
DoD Directive 5000.1 establishes the management process by which the DoD 
provides effective, affordable, and timely systems to the users. DoD . 

Directive 5000.1 states that all acquisition professionals shall continuously 
develop and implement initiatives to streamline and improve the Defense 
Acquisition System. Specifically, this directive states that military decision 
authorities and program managers shall examine and, as appropriate, adopt 
innovative practices that reduce cycle time and cost and encourage teamwork. 
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fFS#S i ) -The TRAILBLAZER J . / Ian initiative that 

began in March 2001 at a cost of approximatel y . [ focused on a set of 

''efforts to demonstrate essential capabilities acd escaonsn &n initial Operational 
Capability, by June 2002 that would lead the transformation of NSA into an era of 
effective and efficient digital network exploitation. Hie immediate objective of 





























\b) (1) 



(TSj'^S^Studfes. Prior to THENTHREAD deployment, the Technology Test and 
Evaluation Assessment, March 25, 2002, advised that the following! - ^ I 



• (U) Generate user, system, technical, and programmatic documentation; 





(U) Management Comments on the Finding and Audit 
Response 

(U/yTOtfO) Management Comments. The NSA management stated that the 
draft report did not include significant information or included wrong or 
misleading information. The complete list of management comments is in the 
Management Comments section of this report 
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(U//SDUO). Audit Response. We obtained, reviewed, and analyzed many 
documents and we interviewed several NSA and contractor personnel to form the 
conclusions in this report. We included all information relevant to support the 
report findings and recommendations. Many people we interviewed asked not to 
be identified for fear of management reprisal. We gathered information from 
many sources to support the finding. Paragraphs frequently contain information 
from several sources to reinforce conclusions. Based on management comments, 
we made various editorial changes to the draft report. See Appendix J for a 
completed audit response to each management comment. 


(U) Recommendations, Management Comments, and Audit 
Response 
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B. The NSA Transformation 



(U) Congressional Concerns on Digital Network Exploitation 

(U) The National Defense Authorization Act for FY 2002 stated that: 

c 403 - fTS/ZSI) - Tbs NSA has clearly made great strides in seeking to 

86-36 transform itself in many areas. However, the NSA appears to have 

made only modest progress In the area most important to its future: 
acquiring the technical ability to operate effectively against the 
emerging global' network. The NSA has long known that 



(U) TRAILBLAZER 


— ( -S //SI) On March 24, 2000, the Director, NSA signed the TRAILBLAZER 1 
charter to create a new Signals Intelligence enterprise to exploit the global 
network. TR ATr.BLA7.HR 1 , with an estimated development cost of 
approximately ~lvas to respond to the opportunities and challenges 

provided by efHHglflg lecnnblogies to design new operational practices for on- 
time delivery of products and services in the manner that best met customer 
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tr 5 trj 



(S^ysi^needs-unres trained by past practices or past technologies. 

TRAILBLAZER 1 had the following key responsibilities 

* (U/fg etlO) Ensure the execution of a new NSA program transforming 
the U.S. Cryptologic System exploitation of the global network. Map 
detailed digital network exploitation mission requirements to a 
cryptologic architecture and implement them; and 

• (U//FWQL Act as the catalyst for transforming customer interaction 
and operational practices. Begin the retooling of NSA 
programmatically, technologically, and culturally to provide mission 
services and capabilities that are responsive to the emerging 
technology and customer needs. 


1 

1 

1 

1 

1 
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8 } NSA Modernization Studies 

" ''(^^irst>JSA Mpdeniization Study. The House Permanent Select Committee 
onlhteliigence diregted-tlie first “NSA Modernization Study,” July 2001, which 
was conducted by The Nafiorral Tcchnology Alliance, National Center for 
Applied Technology. After a thoroughreyiew of ongoing ARtivitity; wittiin,fliR_ — 

\ TRAILB LAZER Program Management Office ! 

\ pnd interviews wiih key program personnel, the National Center for 

Applied Technology team observed that the TR ATLm.AZP.R was noorlv 

executed and had an overly expensive! 

\ The NSA Modernization Study recommended mat: ~ 



Second NSA Modernization Study. A second National Center for 
Applied Technology report, a "quick-look” follow-on report, was completed in 
December 2001. The Chief, Signals Intelligence Programs invited the original 
study team to conduct a follow-on study to evaluate progress within the 
TRAILBLAZER Program Management Office. Despite noting some changes 
made in NSA management and the TRAILBLAZER Program acquisition strategy 
that promised to address the conflicts and issues noted in the first report, the study 
team found that: 
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(U) Conclusion ^ \ 

— (S//SI)~T RAILBLAZBR was develope d to create a new Signals Intelligence 1 \ 

enterprise to exploit the global network) • M 
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(b){l) 

(b) (3) -50 USC 403 
(b) (3) — P.L, 86-36 
\ 


(U) Summary of Management Comments on the Audit Finding 
and Audit Response 

(U//FOtJ©> Management Comments. NS A management referred to several 
statements in the report that did not include all referenced documents and that 
some paragraphs contained wrong or misleading information. The complete list 
of management comments is located in the Management Comments section of 
this report. 

(U//FOCQ) Audit Response. We obtained, reviewed, and analyzed many 
documents to form the basis for the.conclusions in this report We also 
interviewed several NSA and contractor personnel, many of whom asked not to 
be identified for fear of management reprisal. We gathered information from . 
many sources to support the finding. Paragraphs frequently contain information 
from several sources to reinforce conclusions. See Appendix J for a complete 
audit response to each management comment. 


(U) Recommendations, Management Comments, and Audit 
Response /(fa) 

* /(b) (3J-P.L. 86-36 

' / (b) (5) 

(^Revised Recommendation. As a result of management cdmmeSts, we 
revised Recommendations B.l and B.3 to clarify the intended -use of \\ 


1. (U/flf&UO)-We recommend that the Director, National Security 
Agency/Chief, Central Security Service; 


\\ 


ffa Designate a team to asses 


(U// F9UQ) Management Comments. The Director, National Security 
Agency/Cbief, Central Security Service partially concurred with the intent of the 
recommendation but was waiting for the results of a cost and schedule impact 
study before fully concurring. The Program Executive Office is establishing an 
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(U//fCr9G) independent review team to complete the study before Milestone B, 
scheduled for December 2004, because the study will have an effect on both the 
cost and schedule for Milestone B . 



-(G)- Audit Response. Based on management comments, we revised this 
recommendation. The Director, National Security Agency/Chief, Central 
Security Service-comments were responsive and the proposed actions meet the 
intent of the recommendation. The immediate actions taken to designate an 
independent review team. to perform the assessment are commendable. 
Milestone B has slipped to' Febr uary 2005 at the earlie st: therefore . NS A will ha' 
, time to complete the assessment! 

TRAiLBLAZbR. We request that the Director, National Security Agency/Ciuet 
Central Security Service provide a risk mitigation plan for cost and schedule 
requirements and provide comments on a final concurrence. 

2. (U//FOtf0)-Provide the results of the external assessment to the 
TRAIL-BLAZER Technology Demonstration Platform Milestone Decision 
Authority for use in making the Milestone B decision. 

(U//F0HQ£Management Comments. The Director, National Security 
Agency/Chief, Central Security Service concurred. The completion of the 
independent assessment is currently scheduled for December 2004. 

(U//FOtfG). Audit Response. Although the Director, National Security 
Agency/Chief, Central Security Service concurred, the concurrence is based on 
completing the assessment requested in Recommendation B.l. We request that 
the National Security Agency management provide additional comments in 
response to the final report identifying NSA plans to eliminate the cost and 
1 schedule risks for completing the assessment. 


(U//FOifS) Management Comments. The Director, National Security 
Agracy/Chief r Gentral^ecurilyService concurred, and stated that thft.assfssmmf _ 
required in Recommendation B.l will be included inlh^ | 

fof Milestone B, scheduled for December 2004. 

(S//5fy Audit ResponseTAlthojiigh the Director, National Security Agency/Chief, 
Central Security Service concurr^ the- concurrence is based on completing the 
assessment requested in Recommendation B. I .' Based on roana Foment mmmfmts . 
to Recommendation B.2.. this recommendation was revised I 
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Appendix A. Scope and Methodology (U) |\ 3 L 


V 


\\ \\ 

(UtfFOfcf 0)*We reviewed documentation dating from November 2000 through 
October 2003 including background information, mission and operational need 
statements, concepts and operational requirements, and contracting and budget 
documents. Also, we conducted tnterviftWjg with nmoram manappro 
technicians responsible for the acouisitionl , !_ 


(U) We performed this audit from April 2003 through May 2004 in accordance 
with generally accepted government auditing standards. \ \ 

(U/MPOt?0)-Limitation to Scope. Because our objective was limited to \ \ 
evaluating a Defense Hotline allegation relating to the acquisition of the \ 

TRAILBLAZER and THINT HREAD programs, we did not review the I \ 

management control nrogram.l ' > . v 


(U) Use of Computer-Processed Data. Wedidnotusecomputer-processed\ 

; data to perform this audit. ' | 

(U/fFQVQ) Use of Technical Assistance. Audit Follow-up and Technical. \ 
Support Personnel assisted us during the audit. Th e Technical Assessment \ 
Division assisted in the analysis of NSA acquisition! ~| 


(U) General Accounting Office High-Risk Area. The General Accounting 
' Office has identified several high-risk areas in DoD. This report provides 
coverage of the Infrastructure Inefficiencies and Information Technology 
Investments high-risk areas. 

(U) Prior Coverage 

(U//FOUQiDuring the last 5 years, the Inspector General of the NSA issued one 
report on TRAILBLAZER that discussed improperly based contract cost 
increases, non-conformance in the management of the Statement of Work, and 
excessive labor rates for contractor personnel. No prior coverage has been 
conducted on the NSA THINTHREAD Program during the last 5 years. 
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(0) Inspector General, NSA 

(U//FDtH3)JReport No. ST-03-0014, “TRAILBLA2ER 1/SIGINT Programs 
Systems Engineering and Technical Assistance Contract,” September 30, 2004 
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(D) NSA Modernization Follow-Up Study 

/ ' . 


TS7?Si)-The ,f NSA Modernization Study,” July 13, 2001, ai 
Modernization Follow Up Study,” December 2001, stated | 
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(U) General Recommendations. 

' (U//F©U£5) The) 

recommendations! ' : 


J Assessment Study made the following 




(U) For further details: listed belo w are the seven key findin gs and related 
discussions in the THMTHREAD [Report 

(U) Finding 1 Discussion. 

\ - (U//F6UG) There were no technical issues that wou ld prevent THINTHREAD 



(U) Current THINTHREAD system development procedures were 
effective, and a cooperative development effort was proceeding smoothly 
with the pilot site. 


74 








U//FQ &01 Recommendations. The issues previously mentioned would not 


(p) Finding 3 Discussion 

(U) There was minimal user, system, technical, or p r o grammatic documentation. 
/ The existing documentation was insufficient to support the planned deployments, 

• (U) There was no user documentation on how to use the tool. That was 
/ inefficient and could lead to a significant increase in user error. 

Difficulties using the tool can also lead to an increase in support calls that 
drained project resources. 

/ • (U) THINTHREAD did not have a clearly stated, documented set of 

/ mission goals or objectives. There was no program plan, making it 

/ difficult to communicate information about THINTHREAD at a 

/ programmatic level 

/ dehverabies before, during, and after deployment was not available. A 

/ high-level schedule containing the chronology of deployment was 

/ developed, but a detailed schedule was essential for proper resource 

86-36 management, costing, and prediction. 

(U) No installation and maintenance documentation was completed for 
^' ^^itgjsupport personnel, although tho team was working with the pilot site 

to deVelop'ifistallation doc umentation. I nek of instelfafinn and 

maintenance docu mentation) 


(U) There whs no operational support documentation ! 

could adversely affect timely correction 


(U) The effects of minimal documentation could be overcome if an integrated 
team accomplished the development and deployment of THINTHREAD. 
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(0//FO«Q) Recommendations. Develop necessary documentation. That would 
necessitate an increase in the personnel devoted to documenting the system, but is 
essential to the successful deployment and operation of THINTHREAD. The lack 
of this documentation presented a serious risk to the successful deployment of 
THINTHREAD. \ 

(U) Finding 4 Discussion \ 

m/ZFQhie H ^Iovmentplaimi'ng iv w slo w to occ ur], , 


/ * (U//FQUQX.His torically, the THINTHREAD Program Team focused on 

the prototype demonstration at the pilot site. The urogram’s scone did not 


>.)J3)-P.L. 86-36 



(U//FOUO) The THINTHREAD Program Team did not have a 
comprehensive planning process. The lack of planning activities with 

delayed deployment efforts and increased deployment costs. 


relationships between work roles throughout the system’s life cycle were 
uot defined. The authorities for each work role had not been identified. 
The organizational structure set forth by the THINTHREAD Program 
Team focused on system development, and did not address system 
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CU//FQVO) de ployment ! 

- I _ iT he THINTHREAD Program leam was detii 

additional \vork roles necessary for deployment activities. 


• (U) THINTHREAD did not identify all the organizations requiring 
coordination through the various stages of deployment. 

(D/^WytU,Recommendatlons. The findings presented a serious risk to 
successful deployment of THINTHREAD. To mitigate those risks, the following 
recommendations were suggested. 



(U) Finding 5 Discussion 


(U/ /FQUO) -Xhe customer service and user support processes for THINTHREAD 
were manually intensive and directly involved the technical staff. 

• (U//E©HQ) The qualify ofse rvice and s a roport.received "l asing 

.T HB^XHREAB -wigjnconsistentT- Some) (interviewed by 

"Testj Technology and Evaluation Team viewed the THINTHREAD 
„ Pm gram Team to be unresponsive in general customer support, while 

— [e xperienced good respons iveness to their requests for 

additions or modifications } | 

• (U//F0H9) There was no formal trouble ticket mechanism in place. If a 
user or operator had a problem with THINTHREAD, he contacted 
someone on the THINTHREAD Program Team. This contact depends on 
personal acquaintance, because no list of THINTHREAD personnel was 
readily available, and there was no mechanism to locate the proper 
THINTHREAD contact Similarly, the status of the response to a request 
could only be learned by personal contact. 

• (U/7FOtf0) There was no formal median ism for managing externally 
generated requirements, to include submitting a new requirement to 
THINTHREAD or for following the progress of the work. 
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team was in charge of 
"'[ith just a few users, 


maintainin g 1 W ith just a fev 

•—that person was able to_enter.or.cliange | R ecording to 

— requests-received on the telephone or bye-mail. However, as mor 


(b) (3) -P. L. 86-36— 




(U//F0UO) A list of Web interface functionalities proposecT" 
was not incorporated into THINTHREAD. 

(U//F&tJQ) Recommendations. The findings that led to the recommei 
.-Would not have preve nted the successful deployment of THENT ‘ 

I j The recommendations were madejt0-imp,i 

|_iu2Sia| ioflai eihciency of THINTHREAD afletit-was' tiepjpyt 

• (l^InstitateJan automated, formal THINTHREAD customer service and 
^^tiser support process. 


(U/,‘FO&Q) Acquire and jise a trouble ticketing tool, which can manage 
extern ally generated requirements as well as system problems. This tool 
should ] " ' ~~] The THINTHREAD 

Program Team should address high-priority requirements that fit within 


address Web interface functional enhancements 



• Address tl 

Program Team, fo r mamtaininjg 
handle more u ' 

(U) Finding 6 Discussion 

(U/TFOUG) There was no' process for controllinj 
programmatic, and technical performance 
deployed. 


.depth in the THINTHREAD 
THIN THREAD is scaled to 


tonal, 
once 


(U//F&&&) THINTHREAD did not have a clearly stated set of system 
specifications or a method to validate the THINTHREAD concept or 
performance results. Thus, there was no way to determine if the system 
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teU//Fetfe>-was successful or if it failed, which may lead to confusion in 
stakeholders’ understanding of the success ofTH INTHREAD. 

* (U//F©H©)-THINTHREAD had mechanisms for assessing progress and 
status within the system development phase. However, there were no 
systematic, consistent mechanisms for overall program measurement, 
which included identifying test criteria, establishing program schedules, 
and documenting a system baseline to address progress. No feedback 
mechanisms had been identified such as risk management or contingency 
planning. 

> (U//FOU0) Aside from schedule control, there were no tools to measure 
progress towards program objectives because there were no formalized 
program objectives, 

(U//F 0UQ) THINTHREAD did not have Standards of Performance 
established for personnel developing THINTHREAD. If future 
THINTHREAD development crosses organizational groups, a common set 
of Standards of Performance for personnel may become necessary. 

(U//F©ya) There was no configuration management process, w hich could 
pose a risk when reporting system problems and correcting themj | 

In addition, the lack of a configuration management prqcesscould cause 
difficulty when upgrading or deploying versions, and dlso make rollback 
extremely risky. 

(IWF0U9) The fault or wanting detection was automated but rim 

necessary reactions and responses were not.-| 
software provided a simple way to check status or various system 
parameters through a Weft interface. However, when an operator 
recogndzes'a problemrhie must actively initiate corrective measures. 
^Sometimes that action is a simple matter, such as restarting a process. At 
' jijther fimes, physical hardware must be changed. No corrective action 
"could be executed if an operator was not present to initiate a corrective 
action. 



"*'-^<D// TQUO) - T HINTHREAD had no established process for managing 
requirements.^ 

(U/^OUOTReconmicndarions:-The,findings that led to the recommendations 
would not have preven ted the successfui'deploytnent of THINTHREAD to 

■: | ( The recommendations werenrade tp imp rove the 

operational efficiency of THINTHREAD after it was deployed I I 

| | The following is a list of recommendations. 

« (U//EOHOJ Develop processes and acquire automated tools based on 
clearly stated specifications for controlling and measuring the 
programm atic. Operational and technical performance of THINTHREAD 
deploymeh} j 



1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

T 

T 
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.0 

0 

0 

Q 
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/(b)<3)-P.L. 86-36 

• (U// FQUO) Acquire a commercially available configdration\^ntrol 
system to enable versioning, rollback, and configuration, management. 

The system should interface with the trouble ticketing system arid the 
requirements management system to provide"full traceability. \ \ 

• (UZ/FOtJOJ Develop Standards of Performance (if future TtylNTHkEAD 

development crosses organizational groups). - s \ \ 

• (U//F©tf8) Automate re actions' and responses to the existing fault and\ 

warning detection system [X . | V 


• (U/7F0UQ) Establish a require ments management process for \ 

READ that is linked to ] 




(U) Finding 7 Discussion 


he number and skill mix of TH INTHREAD p 


--- — There were shortfalls with the num bers and skills of pers onnel required for 

(bimi-p.L. 86-36“; planning-and executing-instaUatiem) pUso, there -was alack of 

. documentation, providing the necessary maintenance and help .desk support, 

' configuration management; and-planning andjicfaeduling. C ollectively. tho se 

shortcomings c reated a serious risk to successful completion) 


(V//FOV0) Recommendations. The finding presented serious risk to successful 
deployment of THINTHREAD. It was recommended to continue to identify and 

acquire the nu mber and tvnes r>f skilled nersotmel resources required for, — 

documentation.! ~ I 
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Electronic Industries Association (EIA). A group that specifies electrical transmission ! 
standards. The Electronic Industries Association and Telecommunications Industry 
Association have developed numerous well-biown communications standards, including j 
Electronic Industries Association and Telecommunications Industry Association *232 and j 
Electronic Industries Association and Telecommunications Industry Association -449. j 


(P) 
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International Telecommunication Union Telecommunication Standardization 1 
S ecto r (ITU-T). International body that develops worldwide standards for I 

telecommunications technologies. The International Telecommunication Union 1 

Telecommunication Standardization Sector carries out the functions of the former j 

Consultative Committee for International Telegraph and Telephone. \ 

International Organization for Standardization (ISO). International organization that \ 
is responsible for a wide range of standards, including those relevant to networking. The I 
International Organization for Standardization developed the Open System 
Interconnection reference model, a popular networking reference model. 1 
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(U) 


Institute of Electrical and Electronics Engineers (IEEE) 802 3 . Institute of Electrical 
and Electronics Engineers LAN protocol that specifies an implementation of the physical 
layer and the MAC sublayer of the data link layer. Institute of Electrical, and Electronics 
Engineers 802.3 uses Carrier Sense Multiple Access Collision Detect access at a variety 
of speeds over a variety of physical media. Extensions to the I Institute of Electrical and 
Electronics Engineers 802.3 standard specify implementations for Fast Ethernet. 



Open System Interconnection (OSI). International standardization program created by 
International Organization for Standardization and International Telecommunication 
Union Telecommunication Standardization Sector to develop standards for data 
networking that facilitate multi-vendor equipment interoperability. 







Telecommunications Industry Association (TIA). Organization that develops 
standards relating to telecommunications technologies. Together, the 
J Telecommunications Industry Association and the Electronic Industries Association have 

formalized standards, such as Electronic Industries Association and Telecommunications 
71 Industry Association -232, for the electrical characteristics of data transmission. 
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Appendix J. Response to Management 
Comments (U) 


Management Comments. A significant point of departure among Office of 
Inspector General Report assertions and NSA understanding, reflected within NS A’s 



-(G) MatiagementComments . Although NSA will ag ree to conduct an independent 
assessment ofTRAILBLAZElj f/e believe that it will validate all of 

the assertions made„m our responses to tne omce ot inspector General recommendations. . 
Furthermore, we dr^nfaym ffo* r.mrina1 r .tno firmnlaint (W -fraud waste. anA 

\ abuse occurred du ring th fej | 

I |which Was conducted m an open-fashion to provide the best benefit tor 

Agency mission and customers. More importantly, there has been no evidence 
documented during this Office of Inspector General investigation that supports that 

1(b) [3, -P.L. 86-36 

, - fg//SI) Audit Response. The report doc uments that NSA snmtjronroxi m atehj I 

\ million to develop the TRAIL-BLAZER! 1 • 1 









\ \ 

QJIWSUQXMaazgemmt Comments. NSA recommended that the audit report \ \ 

emphasis NSA’s cooperation and responsiveness to Congress. \ \ 



(U/ /FOUQ) - M anagement Comments. NSA recommended that the audit report 
emphasis that the THINTHREAD capability is valued by NSA. 
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Team Members (U) 


(U) The Office of the Deputy Inspector General for Intelligence of the 
Department of Defense, Intelligence-Audit prepared this report. Personnel of the 
Office of the Inspector General of the Department of Defense who contributed to 
the report are listed below. 

(U) 

Thomas F. Gimblc 


(U> 




l 

! 




m 

PerDcDIOj 


-B Oe - Hn 32533.46 


'"(b) (3)-P.L, 86-36 

\ 


(U/^trtJSijrhe development ofthis response Has been coordinated wit? } j and 

the TRAILBLAZER Program Management Office. \ 

RECOMMENDATION: " \ 

1. (U//F0tfG) A formal -Risk Mitigation Plan is not recommended. The PE,0 is 
committed to conducting the assessment. The team is currently being for med, a 
mtgds planned for March, and the team is expected to report to Mr|_ 


iNLT May 31st. The cost and schedule impact of conducting the assessment 


pose very low risk to the Transformational Programs. 


2. (U//F©y€J} Recommend the response, provided as an attachment, be forwarded to 
the DoD OIG as part of the Agency’s consolidated response to the Final Report. 



"Security classification 


NSA STAFF PROCESSING FORM 






MEMORANDUM FOR DEPUTY ASSISTANT INSPECTOR GENERAL FOR 
INTELLIGENCE AUDITS 


SUBJECT: (U//FOU 0 )~D oD OIG Audit Report Requirements for the TRAILBLAZER 
and THINTHREAD Systems, dated December 15, 2004 (Report 05-INTEL- 
03, Project No. D2003AL-0100) - INFORMATION MEMORANDUM 


(U//F0U9Mn response to the DoD OIG request for additional information on the 
TRAILBLAZER and THINTHREAD report, NSA/CSS provides consolidated corporate 
comments from our Signals Intelligence Directorate, Acquisition Organization, and 
Information Technology Infrastructure Services Organization (now the Information 
Technology Directorate). 



-(GhSubsequent to the DoD OIG report, the TRAILBLAZER Milestone has been 
scheduled for late in the fourth quarter of FY05 from the previously scheduled earlier 
MS B. Accordingly, this now allows sufficient time for the conduct of the assessment 






SECURITY CLASSIFICATION 

NSA STAFF PROCESSING FORM 


flF/l , 
cci ori'J 


TO [ EXREG CONTROL NUMBER 

C°S I 

1 KCC CONTROL NUMBER 

DC09-017-05 . 


1 ACTION 
) | APPROVAL 

lx] SIGNATURE 

EXREG SUSPENSE 

^ SUBJECT i 

KCC SUSPENSE 

Defense Report on TRAILBLAZER and THINTHREAD Systems 

| | INFORMATION 

ELEMENT SUSPENSE 

DISTfllHUriON 


PURPOSE: ( G ON-F IDE N'T IAL//2 0 291129) Attached for your review and signature is the NSA 
consolidated response to the report from the DoD OIG on requirements for TRAILBLAZER and 
THINTHREAD systems. 


Enel: 

a/s 



■i: 7SA0-FM-O01 *6465 


■ 3QGID1 3253346 
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Final Report 


fTO'hLt rawcomia mu 

(U// TOGO) Project So. D2tt3AUMM 

(V/fPOm) OoD OIG RtccQnueodatioo BA Provide the rt*oK» of the external 
mtmaeai to a* TRAILBIAZER 3>cfci>ototy De*c*»b»tfoa Pkifcnn MiWtaoe 
Dedifcw Asthority tor Me to matlagtfce M&ftooeBdecfafca. 

f LJ) NSA Rapoott: Concur 

i' 

(0 watbned in (Be «*poc*5 to RwonnKaitttei 0.1, tie FEO tad 
TRAILELAZER Meed to provide tbs taclti of tie fadepeodecl twm-vm to gg 
TRAHJ3LA2ER MBtttoae Dedxlon Authority (MDA) tot -m la nutta* tie Milwtone 
B ifccfeba Ttwe results wlU tin be sent to tie DoD JO, the NSA KJ, sod NSA's 0£5ce 
ofOaenlCotmej. 

(U/9909) It thouid el» be noted thu TRAHBLA2HR has fcnj-esubtabed pUcj tor 
extend tuemeet of tie TOP tor pespntlon tor Mffeaaoe B. He 
VeriScdion ud VlhdiUoa OV4.V) of 29 Evdurioa Criteria Is trie* conducted by. 
KSA't independent Office of Cotpprae Auetsfflemi’ Tat tad TeehricaJ Evaluation 
efetteat (ITB). Tie Euly Opcntioratl Atttssaent rod Opcuieutl Aj*y®aeat nil bo 
performed by ETC, vtlei bid esttbHdxd a presence in TRA1LBLAZER e**$ by 
Novsnter 2003. Rettls of both activitiM ivin be provkfcdto tie Mfflestoce Decision 
Authority before MBestoaeB. 


(U) NSA Sm^obm: Coocot 






















National Security Agency/Central Security 
Service Comments (U) 
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Appendix K. Report Distribution (U) 

(U) 

Office of the Secretary of Defense 

Under Secretary of Defense for Acquisition, Technology, and Logistics 
Under Secretary of Defense (Comptroller) 

Under Secretary of Defense for Intelligence 

Assistant Secretary of Defense for Networks and Information Integration 

Other Defense Organization 

Director, National Security Agency 
Inspector General, National Security Agency 
Inspector General, Defense Intelligence Agency 
Inspector General, National Geospatial-Intelligence Agency 
Inspector General, National Reconnaissance Office 

Congressional Committees and Subcommittees, Chairman and 
Ranking Minority Member 

Senate Subcommittee on Defense, Committee on Appropriations 

Senate Committee on Armed Services 

Senate Select Committee on Intelligence 

House Subcommittee on Defense, Committee on Appropriations 

House Committee on Armed Services 

House Permanent Select Committee on Intelligence 


(U> 


m 
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(b) (3) -50 USC 403 



-(G)- Audit Response. Our wording reflects information received from our sotuces. We 
reviewed all of the comments below and checked our sources and made adjustments as 
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id 

l\b) (3) -SO use 403 
/ (t> K(3)-P.L. 86-36 

iiV_ 



'ftS)JVfanagcment Comments. NSA management comments also questions the 
Executive Summary’s statement that “NS A modified or suppressed/ studies and 


audit report did not identify where the information was obtained. 

(&) Audit Response. We have documented information to support this statement; \ 
however, because of fear of reprisal, we agreed to keep the sources anonymous. \ 

Ma nagement Comments. The Office of Inspector G eneral Rep ort’s citations nf\ 


also incomplete and contained inaccuracies, The presentation 






^\b) (1) 

/ fe) (3) -50 USC 403 
33 &>(3}-P.L. 86-36 

frv 


~^SI).of dollars” is not accurate. As the JpAModemization. Study observed, \ 
TRAILBLAZER is not only about di^taluetwork expl oitation. but_aiso^en&6mr> asses 
cultural transformation (people, processes, and policies! ! • \\ I 

| | Aiid, although TRAILBLAZER is currently in. the 

developmental phase, it has-already delivered significant capabilities. \ \ 

■ (S//S1) A udit Response. The NSA management comments are inaccurate. Hi e renox 
Executive S ummary does not claim that TRAILBLAZER development “waste d; \ 

| (of dollars.” Instead, this phrase is only used to accurately describe the \ 

} speclhc hotline complaint allegation that initiated this audit. \ \ 
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(U//f©£Q)JVfanagemeiit Comments. NSA/management comments stated that the 
TRAILBLAZER has long-established plans for external assessment of the Technology 
Demonstration Platform for preparation for Milestone B. The Independent Verification 
and Validation (TV&V) of 29 Evaluation Criteria is being conducted by NSA’s V \ 
independent Office of Corporate Assessments’ Test and Technical Evaluation element 
(TTE). The Early Operational Assessment/hnd Operational Assessment will be \ \ 
performed by JITC, which had established/a presence in TRAILBLAZER spaces by \ 
November 2003. Results of both activities will be provided to the Milestone Decisioh 
Authority before Milestone B. j \ \ 

(U//fTH?0). Audit Response. TRAILBLAZER postponed Milestone B scheduled 'for 
December 2004 because of technical issues with the Technology Demonstration 
Platform. February 2005 is the earliest date that TRAILBLAZER will be ready for 
MILESTONE B. / 

(U77FOBQ)JVfanageinent Comments. NS A management comments stated that the 
. .. results of fheindencndant assp^s.smcrifo a^ • 

I will be provided to the MDA by Milestone B, 

1 cumaiuy ucawuteo 

Management Comments. NSA management comments s! 

Summary’s statement that TRAILBLAZER development “wasted] 
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ruideltnes of PoD Directive 5000.2. \ 


fb) (1} 

to) (3) -50 use 40: 
(b) (31-P.L. 86-3i 


—(Sj^S^uManagement Comments. NSA management comments stated that the Executive 
Summary’s ass ertion that NSA “disregarded solutions to urgent national security needs” 
isnotaccurateA 


\ — (SftSfy Audit Response. NSA comments are inaccurate. Maior issues raised in the 
\ management comments are addressed in the reoort specifically the issue of ' 


(€) Management Comments. NSA managements stated that TRA1LBLAZER and tjid 
Technology Demonstration Platform have invested considerable effort to build a signal \ 
intelligence architecture that is open and seal cable, applies DoD, commercial, and NSA \ 

n-r\A tTiof io ^AOlimonfarJ irt IJpamAti/nrl* irton/e » 








